Classification of Highway Lane Change Behavior to Detect Dangerous Cut-in Maneuvers

Recognizing dangerous driver behavior is an essential part of predicting accurate vehicle trajectories in vehicle active safety systems. This paper proposes a lane change behavior classification approach to detect dangerous cut-in behaviors on highways. First, a probabilistic lane change behavior classifier is proposed based on Hidden Markov Models (HMMs). Then, time series data of lane changing vehicles from both normal and dangerous driving data sets are analyzed and compared to extract decisive features that are more likely to appear in dangerous lane change processes. A feature detection module is proposed specifically considering decisive features correlated to dangerous lane change. Furthermore, the feature detection module is integrated into the HMM classifier to enhance classification ability. The proposed classifier is verified with a separate test data set, and shows satisfactory results in reducing false negative rate of misclassification. Liu, Kurt, Redmill, and Ozguner 2 INTRODUCTION Vehicle active safety technologies play an important role in improving traffic system safety, capacity, and efficiency. By extending the perception ability of human driver and taking appropriate actions before a potential collision, these technologies could significantly reduce collisions caused by fatigue driving, driver distraction, etc. To achieve this goal, a particular area of focus is understanding the traffic context and accurately predicting motion of surrounding vehicles. For the first portion, probabilistic methods are implemented to handle the scenario uncertainty, such as Dynamic Bayesian Network (1), Gaussian Mixture Model (2), and Hidden Markov Models (HMM) (3). By abstracting observations of surrounding vehicles into symbolic maneuver representations, traffic context identifying models try to predict future status of these vehicles with a semantic formulation that is beneficial to long term predictions. For the second portion, with the predicted maneuver, the future trajectory of a perceived vehicle is generated using trajectory prediction approaches (4–6). However, for a given maneuver, trajectories of a perceived vehicle vary severely in different traffic situations. Especially, in situations where an overtaking vehicle needs to avoid imminent crash in the original lane, the driver will have little time to check the target lane and to maintain a normal lane change trajectory (7, 8). This will cause potential collision with the overtaken vehicle if the active safety system on the overtaken vehicle neglects the dangerous lane change behavior and fails to predict lane change trajectory as a dangerous case. On the other hand, severe reaction of the overtaken vehicle will impact the vehicle convoy following it if all of the lane change trajectories are predicted in an excessively conservative manner (9). Therefore, to accurately predict the lane change trajectory, behavior classification of the lane change maneuver is further brought up, as shown in Figure 2. The basic idea of classifying driver behavior is that vehicle states of risky lane change would be identifiable from expected statistics. This is possible by analyzing naturalistic driving data sets. To capture dangerous driver behavior from vehicle state measurements, HMM is often applied to formulating the driver-vehicle interaction (6, 10). HMM-based behavior classifiers take time series data of vehicle states as model emissions and formulate the driver’s operating process that cannot be directly observed as a hidden Markov chain (10). Gadepally, et al (3) applied HMM-based classifier to driver intention prediction at intersections. Their results showed that HMM-based classifier yeilded lower misclassification rates comparing to a common K nearest neighbors algorithm. Moreover, to determine the parameters of HMMs in the classifier, training data of both normal and dangerous lane changes are required. Vehicle state data from driving simulators are often used as training data (11). The desired vehicle states, especially for crash-imminent cases, can be safely generated by repeatedly running the same traffic scenario on the simulator with selected driver candidates. However, these data sets, to some extent, may lose driving behavior features due to differences of driving environments. One way of compensating the data distortion is to study naturalistic driving data. Aoude et al. (10) classified driver behavior of stop sign violation at intersections based on large scale naturalistic driving data set. The naturalistic driving data showed beneficial properties in validating the classification model. Schlechtriemen et al. (12) studied feature selection of surrounding vehicles in their lane change intention recognition model of a highway scenario, and suggested a subset of features that maximize predictive ability. The advantage of using naturalistic driving data for lane change behavior classification is that the vehicle states naturally retain driving features that may not exist in simulation data. This paper takes further step on behavior feature analysis based on naturalistic driving data sets. Specifically, such Liu, Kurt, Redmill, and Ozguner 3 FIGURE 1 Driver maneuver prediction and behavior classification (lane change case). a) host vehicle is analyzing the status of target vehicle to predict future maneuver of the target vehicle; b) host vehicle predicts different trajectories of the target vehicle according to behavior classification results. features are selected that will most likely indicate dangerous lane change behavior on a highway. The decisive features are further imported to a decisive feature detection module integrated to the HMM classifier to improve its classification ability. The rest of the paper is organized as follows. Section II introduces the lane change scenario for behavior classification and the HMM-based classifier modeling. In section III, a time-series data extraction method is proposed for both normal and dangerous lane change cases. The features of naturalistic driving data is discussed. Section IV discusses the vehicle state data features that will be used in posterior decisive feature detection. Verification of the proposed classification approach are given in section V, followed by the conclusion and future work section. SYSTEMMODELING This section describes highway lane change scenarios that lane change behavior classification is necessary to avoid a potential collision. The HMM model implemented to classify different lane change behaviors is also introduced in this section. Behavior Classifier Framework This paper focuses on detecting dangerous lane change behavior of the surrounding (target) vehicles adjacent to the detecting (host) vehicle within its sensing range, as shown in Figure 2. Two assumptions are made that i) the host vehicle could get access to basic dynamic and kinematic states of the target vehicles, e.g. longitudinal/lateral/angular position, velocity, acceleration, etc, via on-board sensors or vehicle to vehicle communication; ii) the host vehicle cannot directly get the drivers’ status of the target vehicles. The host vehicle first receives vehicle state measurements of target vehicles, then predicts and updates their maneuvers periodically. For each maneuver update, driver behavior classification is processed for interested maneuvers to guide trajectory prediction. Liu, Kurt, Redmill, and Ozguner 4 FIGURE 2 System framework of the behavior classifier with decisive driving feature detection While the behavior of the target vehicles are also affected by their surrounding traffic participants, considering these mediate participants in the host vehicle’s classification system will increase system complexity. Here only the vehicles adjacent to the host vehicle are considered. The system structure of the driver behavior classifier is developed in a cascade form, as shown in Figure 2. The behavior classifier consists of a prior data filtering module, a maneuver predictor, a behavior classifier, and a posterior feature detection module. The maneuver predictor consists of several HMMs of which each states for one driver maneuver, e.g. move forward, turn left, change lane, etc., in the given traffic context. In general, the maneuver predictor is a nonlinear function that maps from the observation sequence of vehicle states to a driver maneuver, shown as follows {x1, . . . ,xK} 7→ m ∈ M , x ∈ X ⊂ RM (1) where M is the maneuver candidate set. The behavior classifier consists of two different HMMs under each interested maneuver in M . Two HMMs state respectively for normal driver behavior and dangerous behavior. Here dangerous driving is defined as driver behaviors with unexpected manners involved of a given maneuver, e.g. fatigue or aggressive driving, emergency obstacle avoidance, etc. (6). Similar to the maneuver predictor, the behavior classifier is a mapping from vehicle state observations to driver behaviors of given maneuver {x1, . . . ,xK}×m 7→ b ∈ B (2) where B := {bi, i ∈ {0,1} : b0 = normal,b1 = dangerous}. The decisive feature detection module receives both time series and indication signals, detects dangerous features, and makes posterior revisions of the classification results. Filtered data from the target vehicle are first processed by the maneuver prediction module. Same data sequence is then classified by the behavior classifier within the predicted maneuver. At last, the classification result is adjusted by the decisive feature detection module with corresponding decisive feature analysis to achieve accurate classification results. One advantage of combining the maneuver predictor with the behavior classifier is that the two modules use the same observation data sequences, and have similar HMM structures and training procedures. For the rest of this paper, specific attentions are paid to the driver behavior classification module distinguishing dangerous lane change behavior from normal cases, i.e. the modules after maneuver prediction shown in Figure 2. Information of maneuver prediction before behavior classification can be found in (3). Liu, Kurt, Redmill, and Ozguner 5 FIGURE 3 HMM applied to model driver-vehicle interaction in highway lane change behavior classifier HMM Based Lane Change Behavior Classifier HMMs can properly formulate the time series vehicle state evolution and the correlated high-level driver state transition. An HMM can be expressed as a tuple λ = {N,M,π,T,e} with N discrete hidden states and observations of dimension M specified by distribution e (10). The hidden states form a finite state Markov process that ruling the observations. The transition probability matrix T states the probability of transiting from one hidden state to another, which is given by Ti j = P(qk+1 = s j|qk = si) (3) Vector parameter π = [π1,π2, . . . ,πN ]T shows the initial state distribution of each state, with πi = P(q1 = si),∀i ∈ [1,N] (4) Furthermore, the probability of observing a certain state emission xk, xk ∈ RM from a hidden state si is given by ei(xk) ∈ e, where e represents a distribution family. Here the observation emission is modeled as Gaussian distribution. Figure 3(a) shows the model structure of HMM for classifying lane change behavior. Figure 3(b) illustrates one realization of the HMM model with its hidden regime states. In general, driver states of the target vehicle during the first stage of lane change, i.e. stage before the target vehicle crossing lane mark, are denoted by the N hidden states of an HMM. A driver is assumed to be within one of these states during this stage. The vehicle state measurements of this stage is treated as the emission sequence of the hidden regime states generated by the driver. With this expression, HMMs representing normal and dangerous lane changes could be trained using labeled driving data, respectively. With the two trained HMMs, for an observed vehicle state sequence, the classification result is given by comparing how well the Liu, Kurt, Redmill, and Ozguner 6 given models fit the observation sequence. The behavior classification problem under the HMM modeling is then turned into identifying the most likely hidden regime sequence using the observed emission sequence. To train HMMs representing normal and dangerous lane change processes, Baum-Welch algorithm (13) is applied with historical lane change observations under both normal and dangerous cases. A more detailed description of data extraction under two cases is given in section 4. Given a set of observation sequences of length K, the algorithm computes the Maximum Likelihood Estimation(MLE) of the HMM parameters, i.e., λ̂ ∗(T,π,e) = argmax λ P(x1, . . . ,xK|λ̂ (T,π,e)) (5) Furthermore, forward algorithm (13), is applied to get the classification result. This is achieved by comparing the likelihood ratio of the two trained models to a threshold. The algorithm defines the probability of observing first k observations with the kth state qk = si given an HMM model λ as αi(k) = P(x1,x2, . . . ,xk,qk = si|λ ) (6) and initializes αi(1) as αi(1) = πiei(x1), i = 1, . . . ,n. (7) Then the probability of subsequent observation can be calculated iteratively as follows α j(k) = [ N ∑ i=1 αi(k−1)Ti j ] e j(xk) (8) The forward probability is returned at k = K as P(x|λ ) = N ∑ i=1 αi(K) (9) By calculating likelihood P(x|λ̂i), i ∈B, using equation 9, the likelihood ratio test for lane change behavior classification is given as logP(x|λ̂b0)− logP(x|λ̂b1) b0 ≷ b1 τ (10) where τ is the threshold. TIME SERIES DATA SEGMENTS In order to get effective classification results in practical application, high quality time series data segments are needed to train the HMMs in the behavior classifier. Since simulation data may degrade features that affect comparatively the behavior classification result, this section focuses on extracting data from naturalistic driving data sets that could give more useful guidance to decisive feature selection and driver behavior classification. Liu, Kurt, Redmill, and Ozguner 7 Data Extraction To build the behavior classifier and to train the HMMs, vehicle state data of both normal and dangerous highway lane change cases are extracted from the SHRP2 sample data set (14) and 100Car data set (15). The lane change cases to train the HMM representing dangerous behavior are extracted from records that are labeled as near-crash events and verified via corresponding baseline video. The extracted data segments are also analyzed and compared to seek decisive features that could classify lane changes of dangerous driver behavior. The variables of the extracted vehicle state data are shown in Table 1. A detailed explanation of the data extraction method can be found in (6). While only the data of lane change scenario is extracted and analyzed for decisive feature detection, the data extraction methods and decisive feature analysis approaches can be easily extended to other highway traffic scenarios. TABLE 1 Variables of Extracted Vehicle State Data Segments, Following SAE J760 Vehicle Coordinate Standard vehicle coordinate x axis y axis z axis variables acceleration acceleration yaw rate velocity position position Naturalistic Driving Data Features The idea of feature detection is to find possible unique pattern emissions from data sets of interested traffic scenarios. Here we focus on time series data of vehicle states, i.e. vehicle states with corresponding time stamps of fixed sampling rate. There are several kinds of data features that could be used for this goal. First, the time series features of the data sequences can be captured. In fact, these features are caught by the Markovian property in the HMM model. Secondly, the variable range difference of normal and dangerous driving data can be treated as another kind of feature classifying different driver behavior. In addition, statistical property of the data sets can also be used to get features for behavior classification. A detailed discussion of the statistical features to enhance driver behavior classification is given in the following section. Moreover, the length of the data segments also affects the model parameters and classification performance of the behavior classifier. A data segment with too short length may lose states in the Markov chain, leading to inaccurate classification results. Long data segments, to some extent, could reduce state missing of the Markov chain. However, data sequences of long interval will increase the training complexity. In this paper, the length of the observation sequence is chose as one second, i.e. 10 samples with sampling rate of 10 Hz. The observation length is selected based on the data from the observation interval including enough features to recognize dangerous driver behaviors while not significantly increasing the model complexity. Though observations of higher sampling rate and longer duration are expected to produce better classification results, they are usually limited by the sensor properties and on-board computational capability. Liu, Kurt, Redmill, and Ozguner 8 longitudinal acceleration [m/s] 0 0.1 0.2 0.3 0.4 0.5 0.6 p d f o f c o rr e la te d n o rm a l d is tr ib u ti o n 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 N +1.0s N +0.9s N +0.8s N +0.7s N +0.6s N +0.5s N +0.4s N +0.3s N +0.2s N +0.1s D +1.0s D +0.9s D +0.8s D +0.7s D +0.6s D +0.5s D +0.4s D +0.3s D +0.2s D +0.1s threshold ετ (a) longitudinal acceleration [m/s] -0.6 -0.4 -0.2 0 0.2 0.4 0.6 p d f o f c o rr e la te d n o rm a l d is tr ib u ti o n